Capability Ratios Predict Nothing∗

Modern approaches to political measurement have generally ignored the importance of out-of-sample predictive performance. This is problematic for two reasons: first, many of the abstractions scholars attempt to proxy for are themselves expectations; and second, the resulting measures are prone to overfitting. We advocate a data-driven approach to measurement based on the train-validate-test paradigm from machine learning. We demonstrate the effectiveness of the approach as it applies to proxying the expected outcome of militarized interstate disputes. The standard proxy for expected dispute outcomes, the ratio of material capability indices, has almost no predictive power. We use ensemble learning to construct a new measure from the same underlying covariates—the Dispute Outcome Expectations score, or DOE—whose predictive power far exceeds that of the standard measure. In replications of 18 empirical studies of international relations, we find that replacing standard capability measures with DOE scores usually improves both in-sample and out-of-sample goodness of fit. ∗We thank Zach Jones and Marc Ratkovic for helpful discussions and advice. Bryan Rooney provided excellent research assistance. We also thank the authors listed in Table 4 for making their replication data publicly available. Replication code and a version history of the project are available at https://github.com/brentonk/crpn. †Assistant Professor, Department of Political Science, Florida State University. Email: [email protected]. ‡Assistant Professor, Department of Political Science, Vanderbilt University. Email: brenton. [email protected].